Skip to content

Feature/system kbd - #3687

Open
JorgeGzm wants to merge 6 commits into
apache:masterfrom
JorgeGzm:feature/system-kbd
Open

Feature/system kbd#3687
JorgeGzm wants to merge 6 commits into
apache:masterfrom
JorgeGzm:feature/system-kbd

Conversation

@JorgeGzm

@JorgeGzm JorgeGzm commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

There is no single way to read a keyboard in NuttX today, and it shows in
the applications.

examples/hidkbd reads a USB HID keyboard as a byte stream.
examples/keyboard reads an upper half keyboard as events. Neither works
with the other, so bringing up a new keyboard means picking the right
example first, and there is no answer for somebody whose keyboard is
neither. examples/lvglterm went further and had three mutually exclusive
input sources, with its own help text explaining that the physical keyboard
options "differ in the data the keyboard device delivers on read(), so pick
the one that matches the hardware". That is the abstraction leaking: a user
had to know the keyboard was USB rather than a matrix in order to compile a
terminal.

This adds system/kbd, which reads any keyboard registered with
keyboard_register(), and converts the terminal and the game to the same
model.

  • system/kbd prints every event a keyboard reports, whatever the
    hardware behind it is. With -i it goes the other way and injects into a
    uinput keyboard, either what it reads from its own stdin or every key of
    another keyboard. So an application can be driven from a serial console
    with no keyboard hardware at all, and a real keyboard and an injected one
    can drive it at the same time, which neither can do on its own since an
    application opens a single device.

  • examples/lvglterm has two input sources now, touch and physical, and
    the physical one does not ask what kind of keyboard it is.

  • games/NXDoom handled KEYBOARD_PRESS and KEYBOARD_RELEASE and
    dropped everything else, so every key reported as KEYBOARD_SPECPRESS
    went nowhere. On the simulator that is already every arrow key, since
    sim_keyboard reports them that way.

    Enter needed handling on both sides, which is worth spelling out because
    it is the whole point of this work: a USB HID keyboard reports it as the
    line feed the key produces, and the simulator reports it as
    KEYCODE_ENTER. Two drivers, two conventions for one key. Both have to
    land on KEY_ENTER, and validating on a single backend is how you end up
    handling only one of them.

This is the application half of the keyboard work. The keycodes and event
types it uses came from apache/nuttx#19586, already merged.

Companion PR, and why CI is red

This has a companion in the kernel repo, apache/nuttx#19592, which updates
the two board configurations that name the option this renames. The two
cannot be green at the same time: CI builds each repo against the other's
master, so this one sees configurations that still name
EXAMPLES_LVGLTERM_INPUT_KBD_USB and _KBD_MATRIX, and the companion sees
an apps master that does not have EXAMPLES_LVGLTERM_INPUT_KBD yet. Both
turn green once both are in.

The failures are limited to the normalisation step of those two
configurations, one line each:

  [1/1] Normalize linum-stm32h753bi/lvglterm_kbda
	modified:   boards/arm/stm32h7/linum-stm32h753bi/configs/lvglterm_kbda/defconfig
        -CONFIG_EXAMPLES_LVGLTERM_INPUT_KBD_USB=y
  [1/1] Normalize esp32s3-m5-cardputer/lvglterm
	modified:   boards/xtensa/esp32s3/esp32s3-m5-cardputer/configs/lvglterm/defconfig
        -CONFIG_EXAMPLES_LVGLTERM_INPUT_KBD_MATRIX=y

Worth flagging for review: no in-tree configuration enables SYSTEM_KBD
yet, so CI never compiles the new application here. The companion PR turns
it on in linum-stm32h753bi:lvglterm_kbda. Until then it was built locally
with the two branches paired, see Testing.

Impact

examples/hidkbd and examples/keyboard stay for now. They are what the
in-tree configurations still name, and removing them has to wait until those
configurations have been moved to system/kbd.

EXAMPLES_LVGLTERM_INPUT_KBD_MATRIX and EXAMPLES_LVGLTERM_INPUT_KBD_USB
are replaced by a single EXAMPLES_LVGLTERM_INPUT_KBD. The two in-tree
configurations that selected either one, on the Linum and on the M5Stack
Cardputer, are updated in the kernel PR. The device path has to be spelled
out for a USB keyboard, since that driver names its devices /dev/kbda
onwards while the option defaults to /dev/kbd0.

EXAMPLES_HIDKBD_ENCODED, NSH_USBKBD and MICROWINDOWS_KBD_RAW all read
a stream of characters rather than events, which now comes from
INPUT_KEYBOARD_BYTESTREAM rather than from the USB HID driver itself. All
three depend on it now, so Kconfig refuses a combination that cannot work
instead of letting the application read events and treat them as text. No
in-tree configuration selects the last two.

The terminal no longer honours cursor keys reported out of band as
ordinary presses of 0x80 to 0x83. The M5Stack Cardputer was the only
driver that did that, and it is converted in the kernel PR, which merges
first.

Nothing else changes for an application that was already reading events.

Testing

Validated on hardware in two independent backends: a USB HID keyboard on a
Linum STM32H753BI, and X11 on the simulator.

system/kbd, reading a real keyboard, event model:

kbd: reading /dev/kbda, keyboard events
press     code  97 'a'
release   code  97 'a'
specpress keycode 7
specrel   keycode 7

Counted rather than eyeballed. Typing normally produced 40 presses and 40
releases, perfectly paired, with no key pressed twice while held, no orphan
release, nothing stuck down at the end, and four keys held at once reported
in the right order. The special keys gave 29 press and release pairs on the
same terms, including two arrows held together.

system/kbd -i, driving a keyboard that does not exist, with a second
instance dumping the same device:

nsh> kbd /dev/ukeyboard &
nsh> kbd -i /dev/ukeyboard
kbd: injecting into /dev/ukeyboard, type to send, Ctrl-D to stop
abc
press     code  97 'a'
release   code  97 'a'
press     code  98 'b'
release   code  98 'b'
press     code  99 'c'
release   code  99 'c'

system/kbd -i with a source, forwarding the USB keyboard into the virtual
one, gave 24 press and release pairs across the crossing with no duplicate,
no orphan and three keys held at once.

examples/lvglterm was driven on the board's LCD from a real USB keyboard,
from the serial console through the uinput bridge, and from both at once
with the USB keyboard forwarded into the same virtual keyboard. The terminal
is the same binary in all three cases and reads a single device.

games/NXDoom was played on the simulator: the menu responds to Enter, the
arrows move, Ctrl fires, Shift runs, Tab opens the map and Escape returns to
the menu. Instrumenting the event handler confirmed the translation, with
Enter arriving as KEYCODE_ENTER and leaving as 13, and the arrows as 5 to
8 leaving as 173 to 176.

Built every configuration in the tree that touches a keyboard, inside the CI
container, with this branch paired against the companion kernel branch so
that the renamed option and SYSTEM_KBD are actually exercised:

### Normalisation
  clean linum-stm32h753bi:lvglterm_kbda     clean olimex-lpc1766stk:hidkbd
  clean esp32s3-m5-cardputer:lvglterm       clean nucleo-h753zi:jumbo
  clean sama5d4-ek:nsh                      clean qemu-intel64:jumbo
### Build
  PASS linum-stm32h753bi:lvglterm_kbda      PASS olimex-lpc1766stk:hidkbd
  PASS esp32s3-m5-cardputer:lvglterm        PASS sim:nxdoom

The CI test lists that cover these boards were also run in full against the
kernel master: arm-08, arm-07 and arm-03, 240 configurations, no
failure.

Two failures seen in the container are environmental and reproduce on an
untouched master: the Xtensa lists fail when the esp-hal-3rdparty
download is truncated, which breaks the .context target for every ESP32-S3
configuration, and ci20 fails to link because the PINGUINOL p32-ld in
the image does not know the target the board asks for:

p32-ld: target elf32-tradlittlemips not found
p32-ld: supported targets: elf32-bigmips elf32-littlemips ...

nxstyle is clean on every file touched.

@casaroli

casaroli commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Please link the dependent PRs. if apache/nuttx#19586 documentation references this tool, then this PR should have been merged before.

Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
Comment thread system/kbd/kbd_main.c Outdated
The hidkbd and keyboard examples do the same thing for one kind of
keyboard each:  hidkbd reads a USB HID keyboard as a byte stream, and
keyboard reads an upper half keyboard as events.  Neither works with the
other, so bringing up a new keyboard means picking the right example
first, and there is no answer for somebody whose keyboard is neither.

Every keyboard registered with keyboard_register() is read the same way,
so one tool covers them all:  USB HID, matrix, simulator, virtio, VNC.

The payload follows INPUT_KEYBOARD_BYTESTREAM rather than a switch of
its own.  An application has no business knowing what hardware is behind
the device, and a build cannot mix the two formats anyway.

The two examples stay for now.  They are what the in-tree configurations
still name, and removing them has to wait until those configurations
have been moved over.

Signed-off-by: Jorge Guzman <jorge.gzm@gmail.com>
i_get_event() handled KEYBOARD_PRESS and KEYBOARD_RELEASE and dropped
everything else, so every key reported as KEYBOARD_SPECPRESS went
nowhere.  On the simulator that is already every arrow key, since
sim_keyboard reports them that way:  the game is unplayable and says
nothing about why.

Handle all four types.  A special key carries a value from enum
kbd_keycode_e rather than a character, so it gets its own translation
into doomkeys.h and contributes no printable character to the event.

The map covers what the game binds by default, which now includes Ctrl,
Shift and Alt.  Fire, run and strafe are on the modifiers, so a keyboard
that reports them is the difference between playing and walking around.

Enter needed folding onto KEY_ENTER as well.  A driver reports it as the
character that it produces, which is the line feed, so it never arrived
as KEYCODE_ENTER and never matched the carriage return that the game
binds the menu to.  Confirmed on hardware with a USB keyboard.

Signed-off-by: Jorge Guzman <jorge.gzm@gmail.com>
The terminal had three input sources to choose from, and its own help
text explained why:  the physical keyboard options "differ in the data
the keyboard device delivers on read(), so pick the one that matches the
hardware".  That is the abstraction leaking.  A user had to know that
the keyboard was USB rather than a matrix in order to compile the
terminal, and swapping one for the other meant rebuilding.

There are two sources now, touch and physical, and the physical one
reads any keyboard registered with keyboard_register().  Which format
arrives is decided by INPUT_KEYBOARD_BYTESTREAM, a property of the build
rather than of the hardware, so the terminal no longer asks.

Cursor keys reported as special events scroll the terminal, which is
what a driver following the current contract sends.  The out of band
codes that the M5Stack Cardputer reports as ordinary presses are still
honoured, so that board keeps working until its driver is converted.

Signed-off-by: Jorge Guzman <jorge.gzm@gmail.com>
Somebody porting a board has to work on the application side before the
keyboard driver exists, and somebody reviewing that work often does not
have the board at hand at all.

With -i the tool goes the other way and writes into a uinput keyboard,
either what it reads from its own stdin or every key of another keyboard.
So an application reading /dev/ukeyboard is driven from the serial
console, or from whatever is on the far end of it, and a real keyboard
and an injected one can drive it at the same time, which neither can do
on its own since an application opens a single device.

Nothing in the application changes:  it is reading a keyboard like any
other, which is the point.

Validated on a Linum STM32H753BI, forwarding a USB HID keyboard and the
serial console into the same virtual keyboard, with the LVGL terminal
reading it.  24 press and release pairs survived the crossing with no
duplicate, no orphan and three keys held at once.

Signed-off-by: Jorge Guzman <jorge.gzm@gmail.com>
The example gated its decoding on HIDKBD_ENCODED, which no longer
exists: the USB HID keyboard driver reports through the keyboard upper
half now, and what produces the byte stream is
INPUT_KEYBOARD_BYTESTREAM.

Left as it was, the option could never be satisfied and the example
would quietly stop decoding special keys.

Reaching the option again brings hidkbd_decode() back into the build
after a spell of being unreachable, and it uses isprint() without
including ctype.h. That is an error rather than a warning under the
-Werror the CI builds with, in the nine configurations that enable
EXAMPLES_HIDKBD, so add the include here rather than in a later commit.

Signed-off-by: Jorge Guzman <jorge.gzm@gmail.com>
Both read a keyboard as a stream of characters:  NSH uses a USB HID
keyboard for stdin, and the raw mode of the Microwindows keyboard driver
decodes the stream with the codec.  That stream now comes from
INPUT_KEYBOARD_BYTESTREAM rather than from the USB HID driver itself.

Say so, so that Kconfig refuses a combination that cannot work instead of
letting the application read events and treat them as text.  No in-tree
configuration selects either option.

Signed-off-by: Jorge Guzman <jorge.gzm@gmail.com>
@JorgeGzm
JorgeGzm force-pushed the feature/system-kbd branch from 669d08a to 4f211f6 Compare August 1, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants